feat: Instruction classes and execution results now support the pickle module #1795

MarquessV · 2024-07-29T16:23:59Z

Description

This closes #1719 (along w/ an additional request to support QPUExecuteResponse), and #1791 (but for all instruction classes).

Waiting on some upstream changes to land, once they do, this PR should pass CI.

Checklist

The PR targets the master branch
The above description motivates these changes.
The change is atomic and can be described by a single commit (your PR will be squashed on merge).
All changes to code are covered via unit tests.
Parameters and return values have type hints with PEP 484 syntax.
Functions and classes have useful Sphinx-style docstrings.
(New Feature) The docs have been updated accordingly.

github-actions · 2024-07-29T16:26:50Z

Bencher

Report	Tue, July 30, 2024 at 17:51:13 UTC
Project	pyquil
Branch	1719-more-pickleable-types
Testbed	ci-runner-linux

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
test/benchmarks/test_program.py::test_copy_everything_except_instructions	✅ (view plot)	10,348,544,208.00 (+0.54%)	10,536,846,752.01 (98.21%)
test/benchmarks/test_program.py::test_instructions	✅ (view plot)	3,896,421,725.20 (-0.73%)	4,051,015,766.66 (96.18%)
test/benchmarks/test_program.py::test_iteration	✅ (view plot)	3,926,822,893.80 (-0.25%)	4,063,221,555.28 (96.64%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

Shadow53 · 2024-07-29T19:50:13Z

test/unit/test_qpu.py

+            # result_data=ResultData.from_qvm(
+            #     QVMResultData.from_memory_map({
+            #         "int": RegisterData.from_i8([[1, 2, 3], [4, 5, 6]]),
+            #         "float": RegisterData.from_f64([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
+            #     })
+            # )


Indeed, removed.

Shadow53 · 2024-07-29T19:54:48Z

test/unit/test_qpu.py

+def test_pickle_qpu_execute_response(mock_encrypted_program):
+    response = QPUExecuteResponse(
+        job_id="some-job-id",
+        _executable=mock_encrypted_program,
+        execution_options=ExecutionOptions.default()
+    )
+    pickled_response = pickle.dumps(response)
+    unpickled_response = pickle.loads(pickled_response)
+    assert unpickled_response == response


Seems like we could have a function that specifically takes Any and does this pickle roundtripping. Not sure if we'll ever add more logic to this, but having it in one place would make that easier.

Sorta like in test/unit/test_quilbase.py

I like that idea, I parametrized the response types over a single test implementation.

jselig-rigetti

This is great, and will be appreciated by many!

Do you know if pickling allow objects to be serialized/deserialized between Python runtimes on separate architectures? For example, if someone were to pickle execution results on ppc64le using pyquil-grpc-web would they then be able to unpickle(?) from arm64 pyquil?

Not a blocking question, just curious.

pyproject.toml

Co-authored-by: jselig-rigetti <97701976+jselig-rigetti@users.noreply.github.com>

MarquessV · 2024-07-30T17:15:13Z

Do you know if pickling allow objects to be serialized/deserialized between Python runtimes on separate architectures? For example, if someone were to pickle execution results on ppc64le using pyquil-grpc-web would they then be able to unpickle(?) from arm64 pyquil?

While that isn't something I have first-hand experience with, the pickling format should be platform-agnostic. There are versioned protocols, but none of them mention platform specificity as an issue.

github-actions · 2024-07-30T17:54:44Z

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
7203	6317	88%	87%	🟢

New Files

No new covered files...

Modified Files

File	Coverage	Status
pyquil/api/init.py	100%	🟢
pyquil/control_flow_graph.py	100%	🟢
pyquil/quilbase.py	94%	🟢
TOTAL	98%	🟢

updated for commit: d9c132e by action🐍

MarquessV · 2024-07-29T18:43:23Z

pyproject.toml

 numpy = "^1.25"
 scipy = "^1.11"
 rpcq = "^3.11.0"
 networkx = ">=2.5"
-qcs-sdk-python = "0.19.0"
+qcs-sdk-python = "0.19.2"
+quil = ">=0.11.2"


Prior to qcs-sdk-python 0.19.2, quil was required to be a specific version. That was meant to ensure compatibility between downstream users of both packages, but in practice that has been over-restrictive. This allows quil to be updated separately from qcs-sdk-python, at least between breaking changes.

MarquessV · 2024-07-29T18:43:49Z

pyproject.toml

@@ -19,12 +19,13 @@ packages = [{ include = "pyquil" }]
 exclude = ["pyquil/conftest.py"]

 [tool.poetry.dependencies]
-python = "^3.9,<=3.14"
+python = "^3.9,<=4.0"


This will close #1793.

pyproject.toml

MarquessV · 2024-07-30T17:41:31Z

test/unit/test_qpu.py

+            # result_data=ResultData.from_qvm(
+            #     QVMResultData.from_memory_map({
+            #         "int": RegisterData.from_i8([[1, 2, 3], [4, 5, 6]]),
+            #         "float": RegisterData.from_f64([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
+            #     })
+            # )


Indeed, removed.

MarquessV · 2024-07-30T17:49:42Z

test/unit/test_qpu.py

+def test_pickle_qpu_execute_response(mock_encrypted_program):
+    response = QPUExecuteResponse(
+        job_id="some-job-id",
+        _executable=mock_encrypted_program,
+        execution_options=ExecutionOptions.default()
+    )
+    pickled_response = pickle.dumps(response)
+    unpickled_response = pickle.loads(pickled_response)
+    assert unpickled_response == response


I like that idea, I parametrized the response types over a single test implementation.

MarquessV added 2 commits July 29, 2024 09:19

feat: Instruction classes now support the pickle module

ddc24ad

Merge branch 'master' into 1719-more-pickleable-types

ac57650

bump qcs-sdk-python, quil

0279642

MarquessV marked this pull request as ready for review July 29, 2024 18:43

MarquessV requested a review from a team as a code owner July 29, 2024 18:44

MarquessV added 2 commits July 29, 2024 11:56

fix other breaking changes

ea9e2af

fix broken tests

b281730

Shadow53 reviewed Jul 29, 2024

View reviewed changes

update lint rules, remove commented code

1bfa5ea

jselig-rigetti approved these changes Jul 30, 2024

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

fix dependency constraint

b510380

Co-authored-by: jselig-rigetti <97701976+jselig-rigetti@users.noreply.github.com>

MarquessV added 3 commits July 30, 2024 10:50

fix tests

3e06bc6

parametrize pickling test

a58d5cb

type hint

d9c132e

MarquessV merged commit ac93417 into master Jul 30, 2024
22 checks passed

MarquessV deleted the 1719-more-pickleable-types branch July 30, 2024 19:48

MarquessV commented Jul 30, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Instruction classes and execution results now support the pickle module #1795

feat: Instruction classes and execution results now support the pickle module #1795

MarquessV commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

Shadow53 Jul 29, 2024

MarquessV Jul 30, 2024

Shadow53 Jul 29, 2024

Shadow53 Jul 29, 2024

MarquessV Jul 30, 2024

jselig-rigetti left a comment

MarquessV commented Jul 30, 2024

github-actions bot commented Jul 30, 2024

MarquessV Jul 29, 2024

MarquessV Jul 29, 2024

MarquessV Jul 30, 2024

MarquessV Jul 30, 2024

feat: Instruction classes and execution results now support the pickle module #1795

feat: Instruction classes and execution results now support the pickle module #1795

Conversation

MarquessV commented Jul 29, 2024 • edited Loading

Description

Checklist

github-actions bot commented Jul 29, 2024 • edited Loading

Bencher

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jselig-rigetti left a comment

Choose a reason for hiding this comment

MarquessV commented Jul 30, 2024

github-actions bot commented Jul 30, 2024

☂️ Python Coverage

Overall Coverage

New Files

Modified Files

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarquessV commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading